Pose-Invariant Face Recognition: Representing Known Persons by View-based Statistical Models

We present a framework for pose-invariant face recognition using parametric linear subspace models as stored representations of known individuals. Each model can be t to an input, resulting in faces of known people whose head pose is aligned to the input face. The model's continuous nature enables the pose alignment to be very accurate, improving recognition performance, while its generalization to unknown poses enables the models to be compact. As a demonstration, recognition systems with two types of parametric linear model are compared using a database of 20 persons. The experimental results showed our system's robust recognition of faces with 50 degree range of full 3D head rotation, while compressing the data by a factor of 20 and more. Pose-Invariant Face Recognition 3 List of Symbols A, S: Analysis and synthesis mapping M: Analysis-synthesis chain mapping SS, T S: Shape and texture synthesis mapping : View-based statistical model LM : Linear PCMAP model PM : Parametric piecewise linear subspace (PPLS) model (:=fLMkg) ~v: Vectorized facial image ~x: Shape vector of a facial image ~ux: Average shape vector of f~xg ~j: Texture vector (Gabor jet) sampled at node n ~unj : Average texture vector of f~j g ~ : 3D angle vector of a head in an image ~u : Average angle vector of f~ g N : The number of facial landmarks Y;B: Shape and texture models F;G;H: Shape-to-pose, pose-to-shape, shape-to-texture transfer matrices P0; S0: The number of principal components included in shape and texture models R: Image reconstruction operator K: Trigonometric functional transformation K: The number of local models in the PPLS model ~ w: Weight vector for the PPLS model k: The k-th Gaussian width of the weight function ~xi: Shape vector estimate by the i-th iteration ~ i: Angle vector estimate by the i-th iteration : Learning rate Pose-Invariant Face Recognition 4